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DETAILED ACTION 
DETAILED ACTION 
Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1-3, 8, 18-20, 25, 28, 29 and 31 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Qiao 6,075,895 in view of Natsuko 6,252,599. 

3. In regard to claims 1 and 18, Qiao discloses a method for determining a gesture 
comprising the steps of: determining a change in a background of an image from a 
plurality of images (see column 1 , lines 64-67 and column 2, lines 1 -5; movement of the 
person in the foreground causes changes in what parts of the background are visible); 
determining an object in the image (see column 2, lines 15-22 and figure 3, step 181); 
determining a trajectory of the object through the plurality of images (see column 3, 
lines 37-39 and figure 1 1 , step 377); and classifying a gesture according to the 
trajectory of the object (see column 3, lines 27-30 and figure 3, step 185). 

Qiao further discloses identifying a gesture according to the trajectory of the 
object. See column 6, lines 58-62, disclosing, "After the pre-processor 152 removes 
(step 181) the background image to generate the player's image, the template matcher 
154 (step 1 83) the player's image to a number of templates to generate template 
outputs". Further see column 8, lines 61-62, disclosing, "the post processor 156 
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analyzes those outputs to identify the specific pre-defined gesture that corresponds" 
and lines 65-67, disclosing "the post-processor tracks (step 377) the player's body 
position". Such tracking through templates of body positions is understood to follow a 
trajectory, as best understood. 

Qiao further discloses determining if the gesture corresponds to a valid command, 
and if the gesture corresponds to a valid command, the workstation automatically 
executing the command. See column 11, lines 11-16, disclosing, "Other approaches 
are applicable, such as a rule-based system... When an input is received, the ruled- 
based system compares it with its rules to generate the appropriate output." Thus, Qiao 
determines if the gesture corresponds to a valid command by comparing an input with 
rules and if the gesture corresponds to a valid command, the command, or "appropriate 
output" is executed. Further, see an example of these steps beginning on line 61 of 
column 11 to column 12, line 12. 

Further in regard to claim 18, Qiao discloses a program storage device readable by 
machine, tangibly embodying a program of instructions executable by the machine to 
perform method steps for determining a gesture (see column 5, lines 9-12). 

Further in regard to claims 1 and 18, Qiao automatically controls a device, as best 
understood, because the gestures of a user automatically control the gesture 
recognizing device, in that the device receives the gestures and maps them, classifies 
them, etc., accordingly without further user control. Thus, the gesture recognizing 
device performs many functions automatically. 
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Qiao does not disclose that the method remotely issues commands to a medical 
imaging workstation or that the commands result in translational and rotational 
manipulation of a medical device. 

Natsuko discloses an invention in which commands are remotely issued to a medical 
imaging workstation and the commands result in translational and rotational 
manipulation of a medical device. See column 6, lines 24-29, disclosing, "The operator 
determines a viewpoint location (a virtual endoscope lens position) in the internal space 
of the internal body tissue and inputs the viewpoint location via the input device 4. A 
FOV centerline direction (an optical axis direction of a virtual endoscope lens) is also 
input via the input device 4." Natsuko further disclosing various input devices that can 
be used as the input device 4, including a keyboard and track ball. See column 13, 
lines 6-18. Natsuko further discloses a command that corresponds to rotation of the 
virtual endoscope. See column 11, lines 50-57, disclosing, "a rotational angle is input 
via the input device 4. (The rotational angle represents a rotational angle around a pivot 
of the endoscope." Natsuko further discloses a command that corresponds to moving 
the virtual endoscope in a right to left direction (translation). Note above that the user 
determines a viewpoint location and FOV (field of view) centerline location. Thus, the 
virtual endoscope can inherently move in a right to left direction if the user commands a 
viewpoint or FOV centerline location that is to the left of the current location. 

Qiao further teaches in column 1, lines 10-23, "In many action-oriented electronic 
games, a player guides... with an input device, such as a low-key-count keyboard, a 
joystick or an electronic pointing device, like a mouse... Though one can learn to 
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manipulate such input devices, it is not natural to act through a joystick or a keyboard. 
An alternative method to control... through gesture recognition. In such a method, the 
player's gesture controls the character's motion." Note that Qiao also discloses in 
column 12, line 30, "The present invention is not limited to playing games." 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the inventions of Qiao and Natsuko, having a virtual 
endoscope that is gesture-controlled. One would have been motivated to make such a 
change based on the teaching of Qiao that input devices such as keyboards and 
joysticks are not as natural to act through as gesture recognition, and since the 
invention of Natsuko can incorporate various input devices. Further, Qiao teaches that 
his invention can be used for applications other than games. 

4. In regard to claims 2 and 19, Qiao discloses the step of determining the change 
in the background comprises the steps of: determining a gradient intensity map for the 
background from a plurality of images and for the current image; determining a 
comparison between the difference and a threshold (see column 2, lines 23-28); and 
determining a pixel to be a background pixel according to the comparison (see column 
2, lines 34-37). Here, generating "the difference between the value of each pixel" is 
understood to mean determining a gradient intensity map because a gradient is 
generated by taking a difference between pixel values. 

5. In regard to claims 3 and 20, Qiao discloses that the object includes a user's 
hand (see column 10, lines 38-51). 
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6. In regard to claims 8 and 25, Qiao discloses the step of identifying the gesture 
comprises the steps of: determining a reference point; determining a correspondence 
between the trajectory and the reference point; and classifying the trajectory according 
to one of a plurality of commands (see column 10, lines 38-51). Here, the "hand 
extended forward" is understood to be a determined reference point. Additionally, this 
limitation can be read on the "rest gesture" and movement away from that position (see 
column 10, lines 52-67 and figure 15). 

7. In regard to claim 28, see rejection of claims 1 and 18. 

8. In regard to claim 29, see rejection of claims 1 and 18. 

9. In regard to claim 31 , see rejection of claims 1 and 18. 

10. Claims 26 and 34 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Qiao 6,075,895 in view of Natsuko 6,252,599 as applied to claims 1 and 18 above, 
and further in view of Funayama et al. 6,332,038 B1 . 

11. In regard to claims 26 and 34, Qiao in view of Natsuko discloses an invention 
similar to that which is claimed in claims 26 and 34. See rejections of claims 1 and 18 
for similarities. Qiao in view of Natsuko does not disclose that determining the object in 
the image comprises the steps of obtaining a normalized color representation for a 
plurality of colors in each image, determining from training images an estimate of a 
probability distribution of normalized color values for an object class, and determining, 
for each pixel, a likelihood according to an estimated probability density of normalized 
color values for the object class. 
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Funayama et al. discloses an image processing device that detects an object in 
an image. See column 2, lines 15-18, disclosing, "For example, when the foregoing 
specified object domain is a human face, by using a probability density function derived 
from a color distribution of human faces, skin areas of faces can be separated from the 
base image." Thus, Funayama et al. determines from training images an estimate of a 
probability distribution of color values for an object class and determines a likelihood 
according to an estimated probability density of color values for the object class. See 
column 14, lines 57-59, disclosing that "the separability measurement process was 
carried out for.. .all pixels of the base image"; thus, the likelihood is determined for each 
pixel. See column 13, line 44, disclosing that "separability is a normalized quantity". It 
is understood that the color values are normalized, as normalization is conventional and 
necessary in order to compare images. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Qiao in view of Natsuko by determining 
an object on the image by obtaining a normalized color representation for a plurality of 
colors in each image, determining from training images an estimate of a probability 
distribution of normalized color values for an object class, and determining, for each 
pixel, a likelihood according to an estimated probability density of normalized color 
values for the object class. One would have been motivated to make such a 
modification based on the teaching of Funayama et al. to use a probability density 
function derived from a color distribution of the object class in order to separate an 
object from an image, thereby determining an object in the image. Also, the 
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normalization of color values is conventional and necessary in order to compare images 
for any purpose, including object determination. 

12. Claims 27 and 33 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Qiao 6,075,895 in view of Natsuko 6,252,599 as applied to claims 1 and 18 above, 
and further in view of Marrin et al. 5,875,257. 

13. In regard to claim 27, Qiao in view of Natsuko discloses an invention similar to 
that which is disclosed in claim 27. See rejection of claim 1 for similarities. Qiao in view 
of Natsuko does not disclose that executing the command further comprises the steps 
of determining the duration of the gesture and correlating the duration of the gesture to 
a intensity and scale in which the command is executed. 

Marrin discloses an invention in which the duration of the gesture is determined 
and correlates to a intensity and scale in which the command is executed. See column 
8, lines 25-28 and 43-46, disclosing, "The tempo dictates the speed at which the 
musical composition is rendered, as well as its timing pattern, and is determined 
primarily from two-dimensional gesture, baton speed (or velocity), and the time between 
beats." Thus, the command is the tempo and the duration of the gesture, which 
correlates in scale and intensity to the speed of the baton movement (since speed takes 
into account the amount of motion in a period of time). Further see column 8, lines 43- 
46, disclosing, "Interpreter 235 may be programmed to detect indication of impending 
tempo increases or decreases (e.g., through analysis of baton speed and direction)". 
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Marrin further teaches that his invention "relates to electronic control, and in 
particular to gestural control of systems capable of exhibiting continuous, dynamically 
changeable behavior." 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Qiao by having executing the command 
further comprise the steps of determining the duration of the gesture and correlating the 
duration of the gesture to a intensity and scale in which the command is executed, as in 
the invention of Marrin. One would have been motivated to make such a change, since 
both inventions "relate to electronic control, and in particular to gestural control of 
systems capable of exhibiting continuous, dynamically changeable behavior." 

14. In regard to claim 33, Qiao further discloses that repetition of a command 
increases the intensity of the command response. It is understood that repetition of a 
command would cause the command to be carried out as many times as the repetition 
occurs. Thus, the intensity of the command response increases accordingly. For 
example, if a command is repeated once, then it is carried out twice and the intensity of 
the command is twice what it would be if not repeated at all. 

15. Claims 30 and 32 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Qiao et al. 6,075,895 in view of Natsuko et al. 6,252,599 B1 as applied to claims 
28, 29 and 31 above, and further in view of Iwamura 6,501 ,515 B1 . 

16. In regard to claims 30 and 32, Qiao in view of Natsuko disclose an invention 
similar to that which is claimed in claims 30 and 32. See rejections of claims 1,18, 28, 
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29 and 31 for similarities. Qiao in view of Natsuko do not disclose that the gesture is 
specifically a rotation of a user's hand or a waving of a user's hand from right to left. 

Iwamura discloses an invention in which the rotation of a user's hand and a 
waving of a user's hand from right to left correspond to gesture commands. See figure 
2, depicting the rotation of a user's hand and figures 1 1 and 16, depicting waving of a 
user's hand from upper right to lower left and from lower left to upper right. 

Iwamure further teaches in column 5, lines 29-52, "The special hand motion is 
not limited to a circular move. Any other special gesture will do... For example, as a 
variation of the circular hand motion, the user 18 may move the hand 20 several times 
(for example twice) toward diagonal direction, for example, lower left to upper 
right... Compared with the circular motion shown in FIG. 2, this is an easier motion for 
the user 18 to make and also easier to detect for the system. A drawback is that such a 
motion is more likely to occur unintentionally than the circular motion and, thus, 
misdetection could occur more frequently... It is a tradeoff." 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the invention of Qiao in view of Natsuko by having 
gestures be a rotation of a user's hand or a waving of a user's hand from right to left, as 
in the invention of Iwamura. One would have been motivated to make such a change 
based on the teaching of Iwamura that "any other special gesture will do" and that the 
selection of the particular gesture for a command is a tradeoff, which is also common 
and conventional knowledge in the selection of gestures for gesture-recognition 
technologies. 
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Response to Arguments 

17. Applicant's arguments filed on May 19,2005 have been fully considered but they 
are not persuasive. 

Applicant's arguments, (see page 7, second paragraph, filed on May 1 9 2005), 
that Qiao action performed by the device is not an exact replication of the detected 
gesture. In the present invention, the gesture is effectively a shorthand instruction that is 
received and interpreted by the device. Furthermore Qiao does not teach or discloses a 
system or method that determines whether a gesture corresponds to a valid command 
and then instructing a device to execute the command resulting in rotation and 
translation of the medical device. 

First, Applicant's claim invention never recites that the gesture is effectively a 
shorthand instruction that is received and interpreted by the device. 

Second, as to Qiao does not teach or discloses a system or method that 
determines whether a gesture corresponds to a valid command. The examiner strongly 
disagrees with Applicant's assertion. Qiao clearly discloses determining whether a 
gesture corresponds to a valid command (see, col.1 1 , lines 11-16, also see, figs. 1 1 and 
15). 

Applicant also argues that Natsuko does not teach or disclose a method for 
automatically remotely issuing commands to a medical imaging workstation. Qiao 
automatically controls a device, as best understood, because the gestures of a user 
automatically control the gesture recognizing device, in that the device receives the 
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gestures and maps them, classifies them, etc., accordingly without further user control. 
Thus, the gesture recognizing device performs many functions automatically. 

Natsuko is cited to teach in which commands are remotely issued to a medical 
imaging workstation and the commands result in translational and rotational 
manipulation of a medical device (See col. 6, lines 24-29, col.11, lines 44-57,6-18). 

Thus, the combination of Qiao's teachings of a gestures automatically controlling 
a device with Natsuko's teachings of a commands remotely issuing to a medical 
imaging workstation clearly reads on applicant's claimed invention. 

Applicant is attacking references individually, one cannot show nonobviousness 
by attacking references individually where the rejections are based on combinations of 
references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1 981 ); In re Merck & 
Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

On page 9, Applicant states that Funayama does not teach a method for 
automatically remotely issuing commands to a medical imaging workstation. Qiao and 
Natsuko are the one which are cited to teach this claim limitation and not Funayama, 
Marrin or Iwamura. 

Conclusion 

21 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Amare Mengistu whose telephone number is (571) 272- 
7674. The examiner can normally be reached on M-F.T-F. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bipin Shalwala can be reached on (571 ) 272-7681 . The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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